Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output head-to-head results in a headToHead.csv file for quick comparison #31

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

l4vr0v
Copy link
Contributor

@l4vr0v l4vr0v commented May 21, 2021

As I test new strategies, parsing results.txt by hand gets rather frustrating. For example, I am looking at two of my strategies- nprtt and ultimateDetective- to understand why nprtt is marginally outperforming ultimateDetective, and this is basically impossible with results.txt. With headToHead.csv, I can just import the file into Excel or Google Sheets and see which strategies nprtt does better on than ultimateDetective (and vice versa) so I know what to dive deeper into, in results.txt.

This Pull Request also does a couple other quality of life improvements:

  • It autoformats prisonersDilemma.py with Black. I'll be honest, this just happened automatically because of my vim autocommand that I was too lazy to disable. It looks cleaner now and I'm kind of hoping you will not make me undo this.
  • It uses decimal.Decimal to represent the average scores within a game (and consequently, overall). Decimal uses fixed-point arithmetic and so doesn't have the imprecision of float. I did this because it makes the headToHead.csv look nicer (floating point imprecision creates some very ugly numbers) without having to give up information by rounding.

You can verify yourself that headToHead.csv is properly parsed by Excel and Google Sheets.

@l4vr0v
Copy link
Contributor Author

l4vr0v commented May 21, 2021

To follow up on the use case mentioned in the first comment, here's an actual head to head sheet and this made it clear to me that uftft was what's breaking my ultimateDetective and causing it to perform unexpectedly worse than nprtt (which makes sense since ultimateDetective tries to detect and exploit a ftft and with a uftft it will instead wind up thinking it's alwaysCooperate, spamming defections, and scoring low).

This feature has significant utility because it made a previously annoying operation really fast.

@l4vr0v
Copy link
Contributor Author

l4vr0v commented May 21, 2021

This has some similarities with #30 but the CSV output in #30 is a different format (albeit a more compact one) that would require a formula to generate a sheet like this. It may make sense to merge both. Or you could merge in #30, close this one, and I can share a spreadsheet template that converts the output of #30 into a head-to-head table like the one generated by this PR.

@l4vr0v l4vr0v mentioned this pull request May 21, 2021
valadaptive pushed a commit to valadaptive/PrisonersDilemmaTournament that referenced this pull request May 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant